Maximum Likelihood Estimation in Log-Linear Models Supplementary Material: Algorithms

We use the theory developed in FR to derive efficient algorithms for extended maximum likelihood estimation in log-linear models under Poisson and product multinomial schemes. The restriction to these sampling schemes is motivated by a variety of reasons. First, these schemes encode sampling constraints that arise most frequently in practice. In particular, these are the sampling schemes practitioners use in fitting hierarchical log-linear models, and especially the class of graphical models. Second, for these particular sampling schemes the log-partition function has a closed form expression and we can easily optimize the associated log-likelihood. Finally, as shown in theorem 9 of FR, the extended MLE of the cell mean value is identical in the two sampling schemes and, for the product multinomial scheme, the estimator is in fact the conditional MLE of the cell means given the sample constraints. Thus these estimates are highly interpretable. Some of the algorithms described in this document are implemented in a MATLAB toolbox available at http://www.stat.cmu.edu/~arinaldo/ExtMLE/. We begin with a high-level overview of extended maximum likelihood estimation, summarizing the theoretical contributions from the previous section and laying down the rationale for the algorithm we propose. To simplify the exposition, we initially develop our result for the simpler case of a Poisson sampling scheme, and later treat the more complex case of product multinomial schemes. Consider a log-linear model with associated d-dimensional log-linear subspace M and design matrix A, which for simplicity we assume to be of full-rank d. (When A is not of full rank, we need only minor changes to the arguments.) We focus on the problem of estimating the cell mean values of the corresponding extended exponential family based on the observed table n. From the results described in section 3.1 of FR, we know that the MLE of μ and, therefore, of m, exists if and only if the observed sufficient statistics t An lie in the interior of the d-dimensional marginal cone CA. In this case, the log-likelihood, parametrized either using the log-linear parameters μ P M or the natural parameters θ P R is a concave function admitting a unique optimizer with finite norm, the maximum likelihood estimate. The MLE does not existent if and only if t P ripF q, for some face F of CA of dimension dF d with associated facial set F . Notice that F , dF and F are random, since they depend on t. Nonexistence of the MLE implies non-estimability of